30 research outputs found
Consistency of Spectral Hypergraph Partitioning under Planted Partition Model
Hypergraph partitioning lies at the heart of a number of problems in machine
learning and network sciences. Many algorithms for hypergraph partitioning have
been proposed that extend standard approaches for graph partitioning to the
case of hypergraphs. However, theoretical aspects of such methods have seldom
received attention in the literature as compared to the extensive studies on
the guarantees of graph partitioning. For instance, consistency results of
spectral graph partitioning under the stochastic block model are well known. In
this paper, we present a planted partition model for sparse random non-uniform
hypergraphs that generalizes the stochastic block model. We derive an error
bound for a spectral hypergraph partitioning algorithm under this model using
matrix concentration inequalities. To the best of our knowledge, this is the
first consistency result related to partitioning non-uniform hypergraphs.Comment: 35 pages, 2 figures, 1 tabl
Newton based Stochastic Optimization using q-Gaussian Smoothed Functional Algorithms
We present the first q-Gaussian smoothed functional (SF) estimator of the
Hessian and the first Newton-based stochastic optimization algorithm that
estimates both the Hessian and the gradient of the objective function using
q-Gaussian perturbations. Our algorithm requires only two system simulations
(regardless of the parameter dimension) and estimates both the gradient and the
Hessian at each update epoch using these. We also present a proof of
convergence of the proposed algorithm. In a related recent work (Ghoshdastidar
et al., 2013), we presented gradient SF algorithms based on the q-Gaussian
perturbations. Our work extends prior work on smoothed functional algorithms by
generalizing the class of perturbation distributions as most distributions
reported in the literature for which SF algorithms are known to work and turn
out to be special cases of the q-Gaussian distribution. Besides studying the
convergence properties of our algorithm analytically, we also show the results
of several numerical simulations on a model of a queuing network, that
illustrate the significance of the proposed method. In particular, we observe
that our algorithm performs better in most cases, over a wide range of
q-values, in comparison to Newton SF algorithms with the Gaussian (Bhatnagar,
2007) and Cauchy perturbations, as well as the gradient q-Gaussian SF
algorithms (Ghoshdastidar et al., 2013).Comment: This is a longer of version of the paper with the same title accepted
in Automatic
q-Gaussian based Smoothed Functional Algorithm for Stochastic Optimization
The q-Gaussian distribution results from maximizing certain generalizations
of Shannon entropy under some constraints. The importance of q-Gaussian
distributions stems from the fact that they exhibit power-law behavior, and
also generalize Gaussian distributions. In this paper, we propose a Smoothed
Functional (SF) scheme for gradient estimation using q-Gaussian distribution,
and also propose an algorithm for optimization based on the above scheme.
Convergence results of the algorithm are presented. Performance of the proposed
algorithm is shown by simulation results on a queuing model.Comment: 5 pages, 1 figur
Non-Parametric Representation Learning with Kernels
Unsupervised and self-supervised representation learning has become popular
in recent years for learning useful features from unlabelled data.
Representation learning has been mostly developed in the neural network
literature, and other models for representation learning are surprisingly
unexplored. In this work, we introduce and analyze several kernel-based
representation learning approaches: Firstly, we define two kernel
Self-Supervised Learning (SSL) models using contrastive loss functions and
secondly, a Kernel Autoencoder (AE) model based on the idea of embedding and
reconstructing data. We argue that the classical representer theorems for
supervised kernel machines are not always applicable for (self-supervised)
representation learning, and present new representer theorems, which show that
the representations learned by our kernel models can be expressed in terms of
kernel matrices. We further derive generalisation error bounds for
representation learning with kernel SSL and AE, and empirically evaluate the
performance of these methods in both small data regimes as well as in
comparison with neural network based models